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ABSTRACT 

The  possibility  of  designing  constrained  adaptive  finite 
impulse  response  digital  filters  is  investigated  as  motivated 
by  a  study  of  adaptive  noise  cancellation.   The  first  con- 
straint considered  consists  of  a  fixed  angle  between  filter 
zeros  and  is  implemented  in  a  master-slave  approach  in  which 
one  of  the  zeros  is  adjusted  adaptively  and  the  others  follow 
subject  to  the  constraint.   The  second  constraint  considered 
is  a  linear  constraint  on  the  filter  weights  and  is  imple- 
mented by  augmenting  the  error  equation  with  Lagrangean 
multipliers.   Simulations  indicate  that  the  approach  is 
feasible. 
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I.   INTRODUCTION 

This  thesis  investigates  the  possibility  of  designing 
constrained  adaptive  finite  impulse  response  (FIR)  filters. 
FIR  adaptive  filters  self -adjust  their  parameters  to  match 
the  output  signal  to  a  desired  signal  in  an  optimal  least 
squares  sense.   Considered  in  detail  is  the  possibility  of 
designing  the  adaptive  process  so  that  either  the  parameters 
(weights)  of  the  filter  are  constrained  by  an  algebraic  for- 
mula or,  alternatively,  are  constrained  so  that  the  zero 
pattern  of  the  FIR  transfer  function  is  fixed  while  the 
actual  positions  of  the  zeros  are  adaptively  adjusted.   An 
application  which  motivated  this  study  is  adaptive  noise 
cancellation  where  noise  rejection  over  a  given  bandwidth 
with  a  specified  frequency  rejection  spectrum  is  required, 
but  the  center  frequency  of  the  rejection  band  is  unknown  and 
needs  to  be  determined  adaptively. 

The  usual  method  of  extracting  a  signal  from  a  strong 
noise  background  is  to  pass  it  through  a  filter  which  tends 
to  suppress  the  noise  while  leaving  the  signal  relatively 
unchanged.   Filters  designed  for  this  purpose  can  be  either 
fixed  or  adaptive.   The  design  of  fixed  filters  is  based  on 
some  prior  knowledge  of  both  the  signal  and  noise  character- 
istics.  Adaptive  filters  have  the  distinct  advantage  of 
being  able  to  adjust  themselves  automatically  and  their 
implementation  requires  little  a  priori  knowledge  of  the 


signal  and  noise  characteristics.   This  type  of  adaptive 
filtering  often  converges  to  the  optimal  filter,  which 
originated  with  the  pioneering  work  of  Wiener  [Ref .  1]  and 
later  was  enhanced  by  Kalman  [Ref.  2]  and  others.   The 
optimal  Wiener  filter  is  defined  as  the  linear  filter  opti- 
mized with  respect  to  a  minimum  mean  squared  error,  where  the 
error  is  the  difference  between  the  filter  output  and  the 
desired  filter  output. 

Widrow  [Ref.  3]  presents  the  classic  FIR  adaptive  filter 
which  is  optimized  via  a  gradient  minimum  seeking  algorithm 
called  the  Least  Mean  Squared  (LMS)  algorithm.   This  chapter 
discusses  the  theory  behind  both  the  Wiener  and  LMS  filters, 
and  demonstrates  that,  for  statistically  stationary  input 
signals,  the  steady-state  values  of  the  LMS  adaptive  filter 
weights  are  accurate  approximations  of  the  Wiener  weights. 

The  following  matrix,  vector,  and  scalar  definitions  are 
used  in  this  thesis.   An  underlined  capital  letter  denotes  a 
matrix  (M) .   A  lower  case  underlined  letter  denotes  a  vector 
(v) .   A  lower  case  letter  which  is  not  underlined  denotes  a 
scalar  (s) .   Finally,  a  capital  letter  which  is  not  under- 
lined denotes  an  internal  element  of  the  corresponding  matrix 
M(n)  . 

In  Chapter  II  the  concept  of  adaptive  noise  cancelling  is 
studied  in  detail.   In  Chapter  III  the  constrained  adaptive 
FIR  filter  is  presented  and  in  Chapter  IV  some  simulation 
results  are  given. 


A  literature  search  has  indicated  that  very  little  re- 
search has  been  done  in  the  area  of  constrained  adaptive 
filters.   Frost  [Ref.  8]  presents  a  constrained  LMS  algorithm 
which  is  capable  of  adjusting  an  array  of  sensors  in  real 
time  to  respond  to  a  signal  coming  from  a  desired  direction 
while  discriminating  against  noises  coming  from  other  direc- 
tions.  A  set  of  linear  constraints  on  the  weights  maintains 
a  chosen  frequency  characteristic  for  the  array  in  the 
direction  of  interest. 

A.   THE  ADAPTIVE  LMS  AND  THE  OPTIMAL  WIENER  FILTERS 

The  LMS  adaptive  filter  shown  in  Figure  1.1  uses  the 
weighted  sum  of  a  set  of  input  signals  which  are  combined  to 
form  an  output  signal  y (n) .   The  input  signal  vector  is 
defined  as 


X(n)   =   [X1(n)   X2(n)   ...   XN(n)]T 


[X(n)    X(n-l)  ...   X(n-N)]T 


(1.1) 


The  weighting  coefficients 


W  -   [Wx    W2    ...    WN]T  (1.2) 


are  the  weights  of  the  system.   Each  input  value  is  multi- 
plied by  a  corresponding  weight  coefficient  and  the  linear 
combination  of  the  sum  of  these  weighted  inputs  forms  the 
output 


•*  y(j) 

FILTER 
OUTPUT 


e(J\ 
ERROR 


DESIRED 

REFERENCE 


Fig.  1.1.   The  LMS  Adaptive  Filter 
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N  N 

y(n)   =    [  W,X. (n)   =    £   WX(n-i)  (1.3) 

i=l    x  i=l   1 

which  can  be  written  in  matrix  form  as 


y(n)   =   X(n)TW   =   WTX(n)  (1.4) 


An  error  signal  is  now  defined  as  the  difference  between 
a  desired  response  d(n)  and  the  actual  response  y(n) . 


e(n)   =   d(n)  -  y(n)   =   d(n)  -  WTX(n)        (1.5) 


The  purpose  of  the  adaptive  algorithm  is  to  adjust  the 
weights  of  the  filter  to  minimize  the  mean-square  error.   A 
general  expression  for  mean-square  error  as  a  function  of  the 
weight  values,  assuming  that  the  input  signals  and  the  de- 
sired response  are  statistically  stationary,  can  be  derived 
in  the  following  manner.   Squaring  the  error  results  in 


e(n)2   =   d(n)2  -  2d (n) X (n) TW  + WTX (n) X (n) TW        (1.6) 


Taking  the  expected  value  of  both  sides  yields 


E[e(n)2]   =   E[d(n)2]  -  2E  [d  (n)  X  (n)  T]  W 


+  WTE[X(n)X(n)T]  (1.7) 


If  the  vector  r   is  now  defined  as  the  cross  correlation 
— xd 

between  the  desired  response  (a  scalar  value)  and  the  input 
vector,  the  result  is 
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r^  =  E[d(n)X(n)]  =  E[d(n)X(n)  d(n)X(n-l)   ...  d(n)X(n-N)]    (1.8) 


The  input  correlation  matrix  R   is  defined  as 


R    =  E[X(n)X(n)A]  =  E 


Xl(n)Xl(n)  Xl(n)X2(n)   ...  Xl(n)XN(n) 
X2(n)Xl(n)   X2(n)X2(n)   ...  X2(n)XN(n) 


\ 


=  E 


Equation  (1.10)  is  usually  written  as: 


XN(n)Xl(n)  XN(n)X2(n)   ...  XN(n)XN(n) 


(1.9) 


X(n)2      X(n)X(n-l)   ...  X(n)X(n-N)  J 
X(n-l)X(n)  X(n-l)2      ...  X(n-l)X(n-N) 


X(n-N)X(n)  X(n-N)X(N-1)  ...  X(n-N) 


J 


(1.10) 


R   (0) 

XX 

R   (1) 

XX 

R   (N) 

XX 

R   (1) 

XX 

R   (0) 

XX 

R   (N-l) 

XX 

R 
—XX 

• 
• 

R   (N) 

XX 

R   (N-l)  ... 
XX 

R   (0) 

XX 

(1.11) 


Now  the  mean  squared  error  (1.7)  can  be  expressed  as 
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E[e(n)2]   =   E[d(n)2]  -  2r  ,W  +  WTR  W 


(1.12) 


Since  the  error  is  a  quadratic  function  of  the  weights 
and  R  „  is  a  positive  definite  matrix  then  the  surface  of  the 
error  has  a  guaranteed  global  minimum.   Gradient  methods 
adjust  the  weights  to  minimize  the  error  by  descending  along 
this  surface  with  the  objective  of  finding  the  bottom. 

The  gradient  V_  of  the  quadratic  error  function  is  obtained 
by  differentiating  Equation  (1.12)  with  respect  to  the  weight 
vector  w. 


=      { 


3E[e(n)H  3E[e(n)^] 


3w. 


3w. 


3M|in)!lT}         (1.13) 


=      -2r    ,    +   2R     w 
— xd  — xx— 


(1.14) 


This  optimal  weight  vector  w  is  called  the  Wiener  weight 
vector  [Ref.  1]  and  is  found  by  setting  the  gradient  of  the 
mean  square  error  function  to  zero. 


W*   =   R_1  r  , 
—      —xx  — xd 


(1.15) 


This  equation  is  a  matrix  version  of  the  Wiener-Hopf 
equation.   The  practical  objective  of  the  adaptive  system  is 
to  find  a  solution  to  (1.15).   An  exact  solution  would  require 
a  priori  knowledge  of  the  correlation  matrices  r  ,  and  R.xx« 
However,  this  information  is  usually  not  available.   Addi- 
tionally, when  the  number  of  weights  is  large  a  direct 
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solution  is  computationally  cumbersome  since  it  requires  an 
N  by  N  matrix  inversion  in  addition  to  the  N(n+l)/2  auto- 
correlation and  cross  correlation  measurements. 

B.   AN  EXAMPLE  OF  THE  WIENER  SOLUTION 

As  an  example  to  illustrate  the  calculations  involved 
for  a  simple  four  weight  Wiener  solution  consider  the 
following  example  using  deterministic  signals.   The  input 
signal  is  the  sampled  sum  of  two  sinusoids  of  different 
frequencies  and  is  given  by 


X(nT)   =   sin(co,nT)  +  sin(w2nT)  (1.16) 


The  desired  value  is  the  sampled  desired  or  reference 
signal. 


d(nT)   =   K  sinU2nT)  (1.17) 


In  order  to  calculate  the  autocorrelation  matrix,  R   ,  note 
that 


Xl(n)      =      X(n)      =      sin(w,nT)    +  sin(u2nT) 


X2(n)      =      X(n-l)      =      sin  (u>    (n-1)  T)    +   sin  (u>2  (n-1)  T) 


X3(n)      =      X(n-2)      =      sin  («-  (n-2)  T)    +  sin  (u>2  (n-2)  T) 


(1.18) 


X4 


(n)       =      X(n-3)       =      sin(u),(n-3)T)    +  sin  (o»2  (n-3)  T) 
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The  following  expected  value  computations  are  taken  over  a 
full  number  of  cycles,  P,  for  both  sinusoids. 


E[Xl(n)Xl(n) ]   =   E[Xl(n)2]   =   Xl(n)2 


-   1+  i  -   1 

2    2      X 


Likewise  all  the  diagonal  terms  will  take  on  the  value  of 


X2(n)2   =   X3(n)2   =   X4  (n) 2   =   1 


Consider  now  the  off  diagonal  terms  of  R 


E[Xl(n)X2(n)  ]   =   Xl(n)X2(n) 


7    (sin  u,nT+sin  u>nnT)   (smio,  (n-1)  T  +  sinu0  (n-1)  T) 
n=0 


Carrying  out  the  indicated  multiplication  yields 


P 
E[Xl(n)X2  (n)  ]   =    \    (sinto,nT)  (sina^  (n-1)  T) 

n=0 


P 
+    I    (sinu)9nT)  (sincj-  (n-1)  T) 

n=0     A  * 


P 
+    I    (sino)nnT)  (sinaj9  (n-1)  T) 
n=0     l 


P 
+    I    (sina)9nT)  (sinw-,  (n-l)T) 
n=0     z 
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The  final  two  terms  of  this  expression  are  zero  leaving 


E  [XI  (n)  X2  (n)  ]   =    I    (sinoj,nTsin  (oj,  nT-co,  T)  +sinw9nTsin  (u0nT-aj0T)  ) 

n=0      L  L  1        A  12. 


Using  the  identity  (sinA) (sinB)  =  j[cos (A-B) -cos (A+B) ]  the 
final  expression  becomes 


E[Xl(n)X2(n)  ]   = 


y  COSU),T  +  y  COSUJ_T 


(1.19) 


Similarly, 


E[Xl(n)X3(n)  ]   = 


y  cos2oj.T  +  2"  cos2cj2T 


(1.20) 


E[Xl(n)X4(n)  ]   = 


j   cos3a),T  +  j   cos3oj2T 


(1.21) 


Denoting 


E[Xl(n)X2(n)  ]   =   a 


E[Xl(n)X3  (n)  ]   =   b 


(1.22) 


E[Xl(n)X4(n)  ]   =   c 


the  autocorrelation  matrix  can  be  expressed  as 


R 
—xx 


1 
a 
b 
c 


a 
1 
a 
b 


b  c 

a  b 

1  a 

a  1 


J 


(1.23) 
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For  the  cross  correlation  matrix  r 

— xd 


rxd   =   E[d(n)Xl(n)   d(n)X2(n)   d(n)X3(n)   d(n)X4(n)]T    (1.24) 


These  terms  compute  as 


E[d(n)Xl(n)  ]   =   K(sinaj,nT  +  sinw2nT)  sinu1nT 


P 
=   K   £   sinio,nT  (sinw,nT  +  sina)2nT) 
n=0 


P  2  P 

=      K      I      sin    u>..nT  +   K      I    (sinoj,nT)  sinu>9nT 

n=0      X       n=0      L  z 


The  final  term  sums  to  zero  leaving 


p 
E[d(n)Xl(n)]   =   K   £   sin2u),nT   =   y  (1.25) 

n=0       L  * 


Similarly, 


E[d(n)X2(n)]      =      K  (sinu,  (n-1)  T  +  sinw2  (n-1)  T)  sinaj,nT 


P 
=      K      I      sinu>,nT(sinu),  (n-l)T) 


n=0 


n=0 


The  final  term  sums  to  zero  leaving 
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P 
+  K  I      sinu),nT(sinu)2  (n-1)  T) 


E[d(n)X2(n)]       = 


_  _ 

K[    I      sin    oj,nT    •  sinw^nTsinco,  T] 

n=0  L  n=0  X  l 


Using  the  (sinA) (sinB)  trigonometric  identity  invoked  for 
(1.19)  and  realizing  that  the  final  term  sums  to  zero,  the 
expected  value  finally  becomes 


E[d(n)X2(n)]       = 


V 

—   cosw,T 


(1.26) 


Likewise, 


E[d(n)X3(n)  ]       = 


|  cos2oj1T 


(1.27) 


and 


K 


E[d(n)X4(n)]      =      j-  cos3A^T 


(1.28) 


The  Wiener  solution  for  w*  as  given  by  (1.15)  is 


w 


*   = 


*1 

-1 

^ 

1 

a 

b 

C 

1 

a 

1 

a 

b 

COSCd,T 

K 

b 

a 

1 

a 

COs2w,T 

2 

c 

b 

a 

1 

cos3u3-,T 

. 

(1.29) 


where  the  values  of  a,  b  and  c  are  given  by  (1.22) .   The 
Wiener  vector  is  calculated  for  the  specific  values  of 

K  =  1 

fl  =  10  Hz 
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f2  =  35  Hz 
T  =  1/256 
For  these  specific  values 
a  =  .8116 
b  =  .3676 
c  =  -.052 
j[cosu)jT]    =    .485 

2-[cosoj,T]    =    .441 

|[coswjT]  =  .3705 
and  the  optimal  Wiener  vector  w*  is 


w: 


w 


*   — 


w 


Vf] 


w1 


(1.31) 


C.   THE  LMS  ADAPTIVE  FIR  FILTER 

•  The  values  for  the  correlation  matrices  are  not  generally 
known  a  priori.   The  LMS  adaptive  algorithm  introduced  by 
Widrow  and  Hoff  [Ref.  1]  is  a  practical  method  for  finding 
close  approximate  solutions  to  (1.15)  in  real  time.   The 
algorithm  does  not  require  measurements  of  correlation 
functions,  nor  does  it  require  matrix  inversion.   The  LMS 
algorithm  is  an  implementation  of  the  method  of  steepest 
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descent.   According  to  this  method,  the  "next"  weight  w(n+l) 
is  equal  to  the  present  weight  w(n)  plus  a  change  proportional 
to  the  negative  gradient.   Thus 

w(n+l)   =  w(n)  -  uV(n) 

The  parameter  y  controls  stability  and  also  the  rate  of 
convergence.   An  estimate  of  the  instantaneous  gradient  V (n) 
is  obtained  by  assuming  that  the  square  of  a  single  error 
sample  e (n)  is  an  estimate  of  the  mean  square  error.   Differ- 
entiating e  (n)  with  respect  to  w  results  in 


~t    .       _       r 3e  (n)          3e (n)  3s(n)     i                                M     ,0» 

V(n)  l~aT7 ~T77 •••       ~TTt ^  (1.32) 

dW.                    dWn  oW 

12  n 

T 

n  trs\       -      i    «.i**i  r3e(n)       3  e  (n)  3e(n)  ,                          n     ,,-x 

V  (n)       =       2    e  (n)  [— rr- — r— ...       —7— J  (1.33) 

3W,               3w0  3W._ 
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The  expression  for  the  gradient  estimate  can  be  approximated 
by 

V(n)   =   -  2e  (n)X(n)  (1.34) 

Using  this  estimate  in  place  of  the  true  gradient  yields  the 
Widrow-Hoff  LMS  algorithm  given  by 

w(n+l)   =  w(n)  +  2pe(n)X(n)  (1.35) 

The  algorithm  is  generally  easy  to  implement  and  al- 
though it  makes  use  of  gradients  of  the  mean  square  error 
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function,  it  does  not  require  squaring,  averaging  or 
differentiation . 

To  show  the  convergence  of  the  Widrow-Hoff  LMS  algorithm 
(1.35)  to  the  Wiener  solution  given  by  (1.15)  write 

w(n+l)   =   w(n)  +  2ye(n)X(n) 


as 


w(n+l)   =   w(n)  +  2yX(n)  [d(n)  -X(n)Tw(n)] 


w(n+l)   =   [I  -2yX(n)X(n)T]w(n)  +  2yX(n)d(n) 


Consider  now  the  ensemble  average.   That  is, 


E[w(n+1)]   =   [I  -2yRxx]E[w(n)]  +2y£xd  (1.36) 


With  an  initial  weight  vector  w(0) ,  j+1  iterations  of 
Equation  (1.36)  becomes 

E[w(j+1)]       =       [I-2yRxx]j+1w(0)   +2uJ     [I"2y^xx3  ^"-xd  d-37) 

This  equation  may  be  put  in  diagonal  form  by  using  the  normal 
form  expansion  of  the  matrix  R   ;  that  is 

Rxx  =   Q^AQ 
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where  A  is  the  diagonal  matrix  of  eigenvalues,  and  the  square 
matrix  of  eigenvectors  is  the  matrix  Q.   Equation  (1.37) 
can  now  be  written  as 


[[w(j+l)]   =   [I-2yQ_1AQ] j+1w(0)  +  2u  f    [I-2uQ"1AQ] 1r  , 

i=0  - 

=      Q-1[I-2uA]  j  +  1Qw(0)   +2y    I     [I-2vQ~1AQ]1r,  (1.39) 

i=0  - 


As  long  as  the  terms  of  the  diagonal  matrix  [I-2yA]  are  all 
of  magnitude  less  than  unity,  then  the  first  term  of  (1.39) 
goes  to  zero  as  the  number  of  iterations  increases.   That  is, 


lim[I-2UA]j+1   -   0  (1.40) 


Writing  (1.40)  as  a  geometric  series  results  in 


lim     [I-2UA]1   =   ~  A-1  (1.41) 

j-*.«  1=o 


or  for  the  specific  component  '  p'  of  the  matrix 


lim  I    [I-2WA]1   =  j±- 


Therefore,  in  the  limit,  Equation  (1.39)  becomes 


limE[w(j+l)]      =      Q_1AQrxd      =      R~£  £xd  (1.42) 

j->00  
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which  is  the  same  as  the  Wiener  solution  shown  in  Equation 
(1.15) . 

Convergence  of  the  mean  of  the  weight  vector  to  the 
Wiener  solution  is  insured  if  and  only  if  the  proportionality 
constant  \i    is  set  within  certain  bounds.   Since  the  diagonal 
terms  of  [I-2yA]  must  all  have  magnitude  less  than  unity, 
and  since  all  eigenvalues  in  A  are  positive,  the  bounds  on  y 
are  given  by 


1  -2yX 

max 


or 


0   <   y   <   1/X  (1.43) 

p      '  max 


where  X     is  the  maximum  eigenvalue  of  R 

max  *  —xx 
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II.   ADAPTIVE  NOISE  CANCELLING  AND  ITS  APPLICATIONS 

In  this  chapter  the  concept  of  adaptive  noise  cancella- 
tion is  considered.   Adaptive  noise  cancelling  is  one  of  the 
most  practical  applications  of  adaptive  signal  processing 
[Refs.  4,5,6,7].   The  basic  principle  involved  is  the  use  of 
a  reference  input  derived  from  one  or  more  sensors  located 
at  points  in  the  noise  field  where  the  signal  is  either  unde- 
tectable or  weak.   The  reference  input  is  adaptively  filtered 
and  subtracted  from  the  primary  input  containing  both  signal 
and  noise  to  generate  an  error  signal  which  controls  the 
adaptive  process.   The  result  is  the  attenuation  or  elimina- 
tion of  the  primary  noise  by  cancellation.   In  circumstances 
where  adaptive  noise  cancelling  is  applicable  levels  of  noise 
rejection  are  often  attainable  that  would  be  difficult  or 
impossible  to  achieve  through  direct  filtering.   Because  the 
concepts  of  adaptive  noise  cancelling  and  their  applications 
are  fundamental  to  constrained  adaptive  filtering  developed 
in  Chapter  III,  they  are  presented  here  in  detail.   For  exam- 
ple, if  the  angle  between  zeros  and  the  magnitude  of  the  zeros 
are  maintained  constant  during  an  adaptive  process,  the  fre- 
quency response  characteristics  remains  invariant.   Specific 
examples  are  discussed  and  the  resulting  equations  indicate 
the  design  limitations  of  these  approaches. 

A.   THE  BASIC  SYSTEM 

Figure  2.1  depicts  the  basic  adaptive  noise  cancelling 
system  concept  .   A  signal  [s]  is  transmitted  over  a  channel 
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Fig.    2.1.      The  Adaptive   Noise   Cancelling   Concept 
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to  a  sensor  that  picks  up  a  noise  [nO]  which  us  uncorrelated 
with  the  signal.   The  combined  signal  and  noise  [s+nO]  form 
the  primary  input  to  the  canceller.   A  second  sensor  re- 
ceives a  noise  [nl]  which  is  also  uncorrelated  with  the 
signal  but  which  is  correlated  in  some  unknown  manner  with 
the  noise  [nO] .   This  sensor  input  provides  the  reference 
input  to  the  canceller.   The  noise  [nl]  is  filtered  to  pro- 
duce an  output  [y]  that  is  as  close  a  replica  as  possible 
to  [nO] .   This  output  is  subtracted  from  the  primary  input 
[s+nO]  to  produce  the  system  output  [z  =  s  +n0  -y]  . 

If  it  were  possible  to  know  the  characteristics  of  the 
channels  over  which  the  noise  was  transmitted  to  the  primary 
and  reference  sensors,  then  it  would  theoretically  be  possi- 
ble to  design  a  fixed  filter  capable  of  changing  [nl]  into 
[nO] .   The  filter  output  could  then  be  merely  subtracted  from 
the  primary  input  to  produce  the  signal  alone.   However,  the 
characteristics  of  the  transmission  paths  are  generally  un- 
known or  known  only  approximately  and  are  seldom  of  a  fixed 
nature,  therefore  precluding  the  use  of  a  fixed  filter. 

In  the  system  shown  in  Figure  2.1  the  reference  input  is 
processed  by  an  adaptive  filter.   Self-adjustment  of  the 
filter's  impulse  response  is  accomplished  by  the  LMS  algorithm 
described  in  Chapter  I.   The  error  signal  used  in  the  adaptive 
process  depends  on  the  nature  of  the  application.   With  noise 
cancelling  systems  the  practical  objective  is  to  produce  a 
system  output  [z  =  s  +n0  -y]  that  is  a  best  fit  in  the  least 
squares  sense  to  the  signal  [s] . 
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Consider  the  system  inputs  shown  in  Figure  2.1  and  the 
filter  output  [y] .   Assume  that  [s]  is  uncorrelated  with  [nO] 
and  [nl]  and  that  [nO]  and  [nl]  are  correlated.   The  output 
[z]  is 


z   =   s  +  nO  -  y  (2.1) 


Squaring  produces 


z2   =   s2  +  (nO-y)2  +  2s(n0-y)  (2.2) 


Now,  taking  the  expectation  of  both  sides  and  using  the  fact 
that  [s]  is  uncorrelated  with  [nO]  and  [y]  produces 


E[z2]   =   E[s2]  +  E[(n0-y)2]  +  2E[s(nO-y)] 

=   E[s2]  +  E[(n0-y)2]  (2.3) 


The  minimum  output  power  is 


Min  E[z2]   =   E[s2]  +  Min  E[(n0-y)2]  (2.4) 


The  signal  power  is  unaffected  as  the  filter  is  adjusted  to 

2  2 

minimize  E[z  ].   When  the  filter  is  adjusted  so  that  E[z  ] 

2 
is  minimized,  E[(nO-y)  ]  is  therefore  also  minimized.   The 

filter  output  [y]  is  then  a  best  least  squares  estimate  of  the 

primary  noise  [nO] .   It  is  also  of  interest  to  note  that 

2  2 

when  E[(nO-y)  ]  is  minimized,  E[(z-s)  ]  is  also  minimized, 

since  from  (2.1) 
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(z-s)   =   (nO-y)  (2.5) 

Adapting  the  filter  to  minimize  the  total  output  power  causes 
the  output  [z]  to  be  a  best  least  squares  estimate  of  the 
signal  [s]  for  the  given  reference  input.   The  output  [z] 

will  contain  the  signal  [s]  plus  noise.   From  (2.1)  the  output 

2 
noise  is  given  by  (nO-y) .   Since  minimizing  E[z  ]  minimizes 

2 
E[(nO-y)  ],  minimizing  the  total  output  power  minimizes  the 

output  noise  power.   Because  the  output  signal  remains  con- 
stant, minimizing  the  total  output  power  maximizes  the  signal- 
to-noise  ratio.   From  (2.3)  it  can  be  seen  that  the  smallest 

2       2 
possible  output  power  E[z  ]  =  E[s  ]  is  achieved  when 

2 
E[(nO-y)  ]  =  0  and  therefore  y  =  nO  and  z  =  s.   In  this  case, 

minimizing  output  power  causes  the  output  signal  to  be  per- 
fectly noise  free.  These  same  arguments  can  be  extended  to 
the  case  where  the  primary  and  reference  inputs  contain,  in 
addition  to  [nO]  and  [nl] ,  additive  random  noises  uncorre- 
cted with  each  other  and  with  [s] ,  [nO]  and  [nl] .  They  can 
also  be  extended  to  the  case  where  [nO]  and  [nl]  are  deter- 
ministic rather  than  stochastic. 

B.   SIGNAL-TO-NOISE  RATIO  IN  THE  ADAPTIVE  FILTER 

At  this  point  it  is  of  value  to  show  analytically  the 
increase  in  signal-to-noise  ratio  of  the  noise  cancelling 
technique. 

As  noted  previously,  fixed  filters  are  generally  inappro- 
priate for  noise  cancelling  situations  because  the  correlation 
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and  cross  correlation  functions  are  usually  unknown  and 
additionally  they  often  vary  with  time.   Adaptive  filters 
"learn"  these  statistics  initially  and  then  track  them  through 
slow  variations.   For  stationary  stochastic  inputs,  however, 
the  steady-state  performance  of  adaptive  filters  closely 
approximates  that  of  fixed  Wiener  filters  and  therefore  Wiener 
filter  theory  provides  a  useful  mathematical  tool  in  care- 
fully analyzing  statistical  noise  cancelling  problems. 

Figure  2.2  shows  the  classic  single-input/single-output 
Wiener  filter.   The  input  signal  is  x(j),  the  output  signal 
is  y(j)  and  the  desired  response  is  d(j).   The  input  and 
output  signals  are  assumed  to  be  discrete  in  time,  and  the 
input  signal  and  desired  response  are  assumed  to  be  statis- 
tically stationary.   The  error  signal  ise(j)  =d(j)  -y(j). 
The  filter  is  linear,  discrete  and  designed  to  be  optimal 
in  the  minimum  mean-square-error  sense.   It  is  considered 
to  be  composed  of  an  infinitely  long,  two  sided  tapped  delay 
line. 

As  shown  in  Chapter  I  (1.15)  the  optimal  Wiener  weight 
solution  can  be  written 


w*(k)   =  Rxx(k)r-xd(k)  (2,6) 


or 


I      w*U)Rxx(k-M   =   rxd(k)  (2.7) 
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To  obtain  the  transfer  function  of  the  Wiener  filter 
consider  first  the  power  spectral  density  of  the  process. 
The  application  of  the  z-transform  to  R   (k)  yields 

2Wafm 


S   (z)   =   z[R   (k)]   =     y  R   (k)z  (2.8) 

xx  —xx         .    L —xx  \  •  / 


Likewise,  the  cross  power  spectrum  between  the  input  signal 
and  desired  response  is 

oo 

Sxd(z)   A   z[rxd(k)]   =    I       £xd(k)z"k  (2.9) 

—  k=-°°  — 


The  transfer  function  of  the  Wiener  filter  is 


w*(z)   =    I      w*(k)z~k  (2.10) 

k=-°° 


For  specific  values  of  an  individual  matrix  the  optimal 
Wiener  transfer  function  can  be  written  as 

S   (z) 

w*(z)  -    r^ir  (2-11} 

xx 

Consider  now  a  single  channel  adaptive  noise  canceller 
with  a  typical  set  of  inputs  shown  in  Figure  2.3.   The  pri- 
mary input  consists  of  a  signal  s(j)  plus  the  sum  of  two 
noises  m0(j)  and  n(j).   The  reference  input  consists  of  a 
sum  of  two  other  noises  ml(j)  and  n(j)  *  h(j),  where  h(j)  is 
the  impule  response  of  the  reference  channel  whose  transfer 
function  is  H(z) .   To  simplify  the  notation  the  transfer 
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function  of  the  noise  path  from  n(j)  to  the  primary  input 
has  been  set  at  unity.   This  procedure  does  not  restrict  the 
analysis  since  a  suitable  choice  of  H(z)  and  of  statistics 
for  n(j)  will  allow  any  combination  of  mutually  correlated 
noises  to  appear  at  the  primary  and  reference  inputs.   The 
noises  mO(j)  and  ml(j)  are  uncorrelated  with  each  other,  with 
s(j)  and  with  n(j)  and  n(j)  *  h(j).   The  noises  n(j)  and 
n(j)  *  h(j)  have  a  common  origin  and  are  correlated  with  each 
other  but  uncorrelated  with  s(j). 

The  noise  canceller  includes  an  adaptive  filter  whose 
reference  input  x(j)  is  ml(j)  +  n(j)  *  h(j)  and  whose  desired 
response  d(j)  is  the  primary  input  to  the  noise  canceller  and 
is  composed  of  s(j)  +  mO(j)  +  n(j).   The  error  signal  e(j) 
is  the  noise  canceller's  output.   When  the  adaptive  process 
has  converged  then  the  optimal  unconstrained  transfer  function 
of  the  adaptive  filter  is  given  by  (2.11),  which  can  be  further 
reduced  as  follows.   The  spectrum  of  the  noise  ml  is 

S  ,  ,(z)  and  that  of  the  noise  n  arriving  via  H(z)  is 

2 

S   (z) |H(z) I  .   Therefore,  the  input  spectrum  to  the  filter 

is 

svv<z>   =   Smlm1 (z)  +  S   (z) |H(z) |2  (2.12) 

xx         mlml       nn 

The  cross  power  spectrum  between  the  filter's  input  and 
the  desired  response  depends  only  on  the  mutually  correlated 
primary  and  reference  inputs  and  is  given  by 
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Sxd(z)   =   Snn(z)H(z"1)  (2.13) 


Substituting  Equations  (2.12)  and  (2.13)  into  (2.11)  results 
in  the  Wiener  transfer  function  and  is  given  by 

S   (z)H(z"1) 

w*(z)   =   — j  (2.14) 

S,,  (z)  +S  n(z)  |H(z)  r 
mlml      nn    '     ' 

The  result  is  a  transfer  function  which  is  independent  of 

the  primary  signal  spectrum  S   (z)  and  of  the  primary  uncorre- 

lated  noise  spectrum  S  n    «(z). 

mOmu 

An  interesting  special  case  which  clearly  brings  out  the 
function  of  the  adaptive  noise  canceller  is  when  the  additive 
noise  ml  in  the  reference  input  is  zero.   Then  S  ,  , (z)  is 
zero  and  the  optimal  transfer  function  becomes 

w*(z)   =   1/H(z)  (2.15) 

That  is,  the  adaptive  noise  canceller  causes  the  n(j)  noise 
to  be  perfectly  nulled  at  the  noise  canceller  output.   The 
primary  uncorrelated  noise  mO(j)  remains  uncancelled. 

Consider  an  analysis  of  the  performance  of  the  adaptive 
noise  canceller  in  terms  of  the  ratio  of  the  signal  power 
density  at  the  output,  p   t(z)'  to  the  noise  power  density  at 
the  primary  input  p   . (z) .   This  ratio  can  be  written  as 


(    \  (    \  (    )  primary  noise 

pout  '  nntz)    mOmOUJ      pwr  spectrum         (2  lgv 

p       .  ( z )  S  (z)      output  noise 

pri  output  noise        «.—  ««,*,4.— ,-, 

F  e  pwr  spectrum 
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with  the  signal  power  spectrum  factored  and  cancelled  out  of 
the  numerator.   Figure  2.3  shows  that  the  output  noise 
spectrum  consists  of  the  sum  of  three  components — one  due  to 
the  propagation  of  mO(j)  directly  to  the  output,  another  due 
to  the  propagation  of  ml(j)  to  the  output  via  the  transfer 
function  W(z),  and  another  due  to  the  propagation  of  n(j) 
to  the  output  via  the  transfer  function  1  -H(z)W(z).   The 
output  noise  power  spectrum  can  then  be  written  as 


S   .   .       (z)   =  S  n  n(z)  +S  ,  ,(z)|w*(z)| 
output  noise         mOmO      mlml    '      ' 


+  S   (z)  |1  -H(z)W*(z)  |2         (2.17) 

nn    '  ' 


Now,  if  the  ratios  of  the  spectra  of  the  uncorrelated  noises 
to  the  spectra  of  the  correlated  noises  (noise-to-noise  den- 
sity ratios)  at  the  primary  and  reference  inputs  are  defined 
as 

Smrimn(z) 
A(z)   =   ™0m0{    .  (2.18) 

Snn(z) 

B(Z)   =  -^± j  (2.19) 

snn(z)  |h(z)  r 

nn    ' 
then  the  transfer  function  of  (2.14)  can  be  written  as 

W*(z)   =   l/[H(z)  (B(z)+1)]  (2.20) 

and  the  output  noise  power  spectrum  (2.17)  can  be  written 
as 
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S  ,  ,  (z) 
S   t   „    lM(z)   =   S  A  A(z)  +      mlml 


output  noise         mOmO       I  H  (z)  I  2  (B  (z) +1)  2 


+  S   (z)  11  - 


nn    '     B  (z) +1 


=   Snn(z)*(z)  +  Snn(z)  BlifrT       (2'21» 


and  the  ratio  of  the  output  to  the  primary  input  noise  power 
spectra  given  in  (2.16)  is 


p   4_(z)      S   (z)  [1  +  A(z)  ] 
out         nn 


P   •  (z)      S         .   (z) 
pri         output  noise 


[1  +  A(z)  ] 


A(z)  +  (B(z))/(B(z)+l) 
_   [A(z)  +  1] (B(z)  +  1] 


A(z)  +  A(z)B(z)  +  B(z)  (2.22) 


This  expression  allows  an  estimation  of  the  level  of  noise 
reduction  to  be  expected  with  an  ideal  noise  cancelling  sys- 
tem.  In  such  a  system  the  signal  propagates  to  the  output 
with  a  transfer  function  of  unity.   From  (2.22)  it  can  be 
seen  that  the  ability  of  a  noise  cancelling  system  to  reduce 
noise  is  limited  by  the  uncorrelated-to-correlated  noise  den- 
sity ratios  at  the  primary  and  reference  inputs.   The  smaller 
in  magnitude  are  A(z)  and  B(z),  the  greater  will  be 

p   . (z)/p   . (z)  and  the  more  effective  the  action  of  the 
out   '  pri 

canceller.   The  desirability  of  low  levels  of  uncorrelated 
noise  in  both  inputs  is  made  even  more  evident 
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by  considering  the  approximations 


!,      small  A(z,  Jsffltj!}     ,     L  +  $*L  (2.23) 


pri 

pout(2) 
pri 


2)   sman  b(„       ^t_  a  i+^ii.  (2.24) 


pout(z)       -  1 

3)       small   A(z)  p         (z)      =      A(z)    j   B(z)  (2*25) 

and  B(z)  pprilz;  Au;         b*z; 


Infinite  improvement  is  implied  by  these  relationships 
when  both  A(z)  and  B(z)  are  zero  resulting  in  complete  re- 
moval of  noise  at  the  system  output  and  perfect  signal  repro- 
duction.  When  both  A(z)  and  B(z)  are  small  other  factors 
such  as  misadjustment  caused  by  gradient  estimation  noise  in 
the  adaptive  process  and  the  finite  length  of  the  adaptive 
filter  limit  system  performance.   These  factors  are  discussed 
at  some  length  in  [Ref .  3] . 

C.   SIGNAL  PROPAGATION  IN  THE  REFERENCE  INPUT 

If  it  is  reasonable  to  consider  reference  noise  propaga- 
tion into  the  primary  signal  input  it  is  also  reasonable  to 
consider  certain  instances  when  the  signal  propagates  to  the 
reference  input.   The  system  depicting  the  adaptive  noise 
canceller  with  signal  components  in  the  reference  input  is 
shown  in  Figure  2.4.   The  derivation  which  discusses  how  much 
of  the  signal  is  cancelled  when  a  portion  of  the  signal  input 


37 


Mo(i) 


Fig.  2.4.   Adaptive  Noise  Canceller  With  Signal 
1  Components  in  the  Reference  Input 
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"leaks"  into  the  reference  input  omits  the  additive  uncorre- 
cted noises  mO(j)  and  ml(j)  in  order  to  simplify  the 
analysis . 

Given  that  the  spectrum  of  the  signal  is  S   (z)  and 

o  S 

that  of  the  noise  is  S   (z) ,  then  the  spectrum  of  the 

nn  c 

reference  input  is  given  by 


Sy„(z)   =   S   (z) |l(z) |2  +  S   (z) |H(z) |2      (2.26) 
aa  ss  nn 


The  cross  spectrum  between  the  reference  and  primary  inputs 
is 


S  ,(z)   =   S   (z)I(z"1)  +  S  „(z)H(z"X)        (2.27) 
xd         ss  nn 


When  the  adaptive  process  has  converged,  the  Wiener  trans- 
fer function  of  the  adaptive  filter  given  by  (2.11)  is 

S   (z)I(z-1)  +  Snri(z)H(z_1) 
W*(z)   =   -=S UR (2.28) 

S   (z)  |l(z)  T  +  S   (z)  |H(z)  r 
ss    '     '      nn    '     ' 

The  transfer  function  of  the  propagation  path  from  the 
signal  input  to  the  noise  canceller  output  is  l-I(z)W*(z) 
and  that  of  the  path  from  the  noise  input  to  the  canceller 
output  is  l-H(z)W*(z).   The  spectrum  of  the  signal  component 
in  the  output  is  thus 

Sss  out  =  Sss^l1  "  *<«>*•<">  I2 


ss 


H(z)-Kz)  |Sriri(z)H(z"1) 
1  nn 


So  (z)  |l(z)  |2  +S   (z)  |H(z)  |2 
ss    '     '     nn    ' 
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(2.29) 


and  likewise,  that  of  the  noise  is 


nn  out 


=   S„„  (z)  1  -  H(z)W*(z) 
nn    ' 


=   S   (z) 
nn 


[I(z)-H(z)]Sc<=(z)I(z"1) 


Sss(z)  |l(z)  |2  +Snn(z)  (H(z))2 


(2.30) 


The  output  signal-to-noise  density  ratio  is  therefore 


pout(z) 


Sss(2) 
Snn<2> 


Snn(z)  HCz"1) 


Sss(z)  Kz"1) 


S   (z)  H(z) 
nn | 

Sec(z)  |l(z) 

ss    ' 


(2.31) 


The  output  signal-to-noise  density  ratio  can  be  conveniently 
expressed  in  terms  of  the  signal-to-noise  density  ratio  at 
the  reference  input  in  the  following  manner.   The  spectrum 
of  the  signal  component  in  the  reference  input  is 


Sec  ^      =   S   (z)  I(z) 
ss  ref       ss 


(2.32) 


and  that  of  the  noise  component  is  likewise 


S     . (z)   =   S   (z) |H(z) 
nn  ref         nn    ' 


(2.33) 


Therefore,  the  signal-to-noise  density  ratio  at  the  reference 
input  is  thus 
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S   (z)  |l(z)  |2 

pref   =   — — ; n  (2-34) 

snn(z)  h(z)  r 

nn    '     ' 
Comparison  of  Equation  (2.34)  with  (2.31)  shows  that 


pout(z)   =  1/pref(z)  (2-35) 


This  shows  that  if  the  noises  in  the  primary  and  reference 
inputs  are  mutually  correlated,  the  signal-to-noise  density 
ratio  at  the  noise  canceller  output  is  simply  the  reciprocal 
at  all  frequencies  of  the  signal-to-noise  ratio  at  the  reference 
input.   That  is,  in  order  to  obtain  a  good  signal-to-noise 
density  ratio  at  the  filter  output  there  should  be  very  little 
signal  at  the  reference  input. 

The  final  objective  of  the  analysis  is  to  derive  an 
expression  for  the  spectrum  of  the  output  noise.   As  with  the 
previous  analysis  it  is  instructive  to  first  write  the  trans- 
fer function  for  the  path  from  which  the  noise  n(j)  propagates 
to  the  output. 

S   (z)I(z"1)+S   (z)H(z"1)     2 

l-H(z)W*(z)   =   1  -H(z)   — 5-S2 2 

S   (z)  |l(z)  T+S   (z)  |H(z)  r 
ss  nn 


S   (z)I(z"1)  [I(z)-H(z)] 
ss 

S(z)  |l(z)  |2+S   (z)  |H(z)  |2 
ss         ■    nn 


When  I(z)  is  small  (2.3  6)  reduces  to 


(2.36) 


41 


1  -  H(z)W*(z)   = 


-Soe(z)I(z"1) 
ss 

S   (z)H(z"1) 
nn 


(2.37) 


The  output  noise  spectrum  is 


S   .   .         =   S   (z) ll  -  H(z)W*(z; 
output  noise      nn    ' 


(2.38) 


Again,  considering  the  case  where  I(z)  is  small  results  in 


Soutput  noise(z)   ~   Snn(z) 


S   (z)I(z"1) 
ss 


SfzjHtz"1) 
nn 


(2.39) 


If  Equation  (2.39)  is  written  in  terms  of  the  signal-to-noise 
density  ratios  at  the  reference  and  primary  inputs,  where 
the  signal-to-noise  density  ratio  at  the  reference  input  is 
given  by  Equation  (2.34)  and 


P   •  (z) 
pn 


A    SS 


S   (z) 
nn 


(2.40) 


then  Equation  (2.39)  can  be  written  as 


S      .         =   S   (z)  I  p   _(z)  I  |p   .  (z)  I 
output  noise      nn    |Href    ' |Hpri    ' 


(2.41) 


Equation  (2.41)  shows  that  the  output  noise  spectrum 
acts  according  to  three  factors  (given  that  I(z)  is  small). 
First,  the  output  noise  spectrum  depends  on  the  input  noise 
spectrum.   Second,  if  the  signal-to-noise  density  ratio  at 
the  reference  input  is  low,  the  output  noise  will  be  low; 
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that  is,  the  smaller  the  signal  component  feeding  into  the 
reference  input,  the  better  the  cancellation  of  the  noise. 
This  is  to  be  expected  and  was  already  shown  by  Equation 
(2.35).   The  third  factor  implies  that  if  the  signal-to-noise 
density  ratio  in  the  primary  input  (the  desired  response  of 
the  adaptive  filter)  is  low,  the  filter  will  be  trained  most 
effectively  to  cancel  the  noise  rather  than  the  signal  and 
therefore  the  output  noise  will  be  low. 

As  an  illustration  of  the  level  of  performance  attainable 
in  practical  situations  consider  the  following  example.   An 
adaptive  noise  cancelling  system  is  designed  to  pass  a 
plane-wave  signal  received  in  the  main  beam  of  an  antenna 
array  and  to  reject  strong  interference  in  the  near  field  or 
in  a  minor  lobe  of  the  array.   Assume  that  the  signal  and 
interference  power  spectra  are  overlapping  and  that  the 
interference  power  density  is  twenty  times  greater  than  the 
signal  power  density  at  the  individual  array  elements.   Then 
the  signal-to-noise  ratio  at  the  reference  input  p   f  is 
1/20.   Assume  also  that  because  of  array  gain  the  signal  power 
equals  the  interference  power  at  the  array  output  which  forms 
the  primary  input  to  the  adaptive  system.   The  signal-to-noise 
ratio  at  the  primary  input  is  p   .  =  1.   After  convergence 
the  signal-to-noise  ratio  at  the  system  output  will  be 


p     =   i/p     =   i/_L  =   20 
Hout      '  ref       20 


If  signal  distortion  is  defined  as 
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pref(z> 

D(z)   "  ^TZT  (2-42) 

pri 

then  the  maximum  signal  distortion  will  be 

D(z)   =   (1/20) /l   =   5  percent 

The  adaptive  cancelling  improves  the  signal-to-noise 
ratio  twentyfold  while  introducing  only  5  percent  distortion. 
Additionally,  the  adaptive  filter  will  provide  the  same  per- 
formance when  the  input  conditions  change  and  a  new  set  of 
convergent  weights  have  been  obtained. 

D.   NOISE  CANCELLING  APPLICATIONS 

This  section  describes  several  practical  applications 
which  demonstrate  the  applied  concepts  of  adaptive  noise 
cancelling.   These  applications  include  cancellation  of  noise 
in  speech  signals,  cancellation  of  antenna  sidelobe  inter- 
ference, cancellation  of  60-Hz  interference  and  cancellation 
of  either  periodic  or  broadband  interference  when  no  reference 
is  available. 

A  prime  example  of  noise  contaminated  speech  is  that  of  a 
pilot  communicating  by  radio  from  the  cockpit  of  an  aircraft 
where  a  high  level  of  engine  noise  is  interfering  with  the 
pilot's  voice.   The  noise  contains,  among  other  components, 
strong  periodic  mixtures  that  occupy  the  same  frequency  band 
as  speech.   These  components  cannot  be  "low  filtered"  or 
"high  filtered"  out  of  the  speech  pattern  and  are  picked  up  by 
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the  microphone  into  which  the  pilot  speaks,  severely  inter- 
fering with  the  intelligibility  of  the  radio  transmission. 
It  is  impractical  to  process  the  transmission  with  a  conven- 
tional fixed  filter  because  the  frequency  and  intensity  of 
the  noise  components  vary  with  engine  speed  and  load  and  even 
the  position  of  the  pilot's  microphone.   By  placing  a  second 
microphone  at  a  suitable  location  in  the  cockpit,  a  sample 
of  the  ambient  noise  which  is  free  of  the  pilots  speech  can 
be  obtained.   This  sample  can  be  filtered  and  subtracted  from 
the  transmission,  significantly  reducing  the  interference. 

Widrow  et  al . ,  [Ref.  4]  demonstrated  the  feasibility  of 
cancelling  noise  in  speech  signals  by  conducting  a  number  of 
experiments  which  simulated  the  cockpit  noise  problem.   Figure 
2.5  shows  the  system  used  for  cancelling  the  noise  in  the 
cockpit  noise  simulations.   A  voice  input  from  a  room  with 
strong  acoustic  interference  was  used  as  the  primary  input. 
A  second  microphone  was  placed  in  the  room  away  from  the 
speaker  and  this  was  used  as  the  reference  input.   The  output 
of  the  noise  cancelling  was  then  monitored  by  a  remote  listener, 
The  canceller  included  an  adaptive  filter  with  16  weights 
whose  values  were  digitally  controlled  by  a  computer.   A 
typical  experiment  used  an  audio  frequency  triangular  wave 
containing  many  harmonics  as  interference.   Because  of  multi- 
path  effects  the  amplitude  and  phase  of  the  interference 
varied  from  point  to  point  in  the  room.   The  noise  cancelling 
system  was  able  to  reduce  the  output  power  of  the  interference, 
which  otherwise  made  the  speech  unintelligible,  by  20  to  25 
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Fig.  2.5.   Cancelling  Noise  in  Speech  Signals 
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dB,  rendering  the  interference  barely  perceptible  to  the 
remote  listener.   No  noticeable  distortion  was  introduced 
into  the  speech  pattern.   Convergence  times  were  on  the  order 
of  a  few  seconds  and  the  system  was  readily  able  to  readapt 
when  the  position  of  one  or  both  microphones  was  changed  or 
when  the  frequency  of  the  interference  was  varied  over  the 
range  of  100  to  2000  Hz. 

E.   CANCELLING  ANTENNA  SIDELOBE  INTERFERENCE 

Another  type  of  noise  cancelling  is  that  of  eliminating 
strong  unwanted  signals  which  are  incident  on  the  sidelobes 
of  an  antenna  array.   These  interferences  can  severely  retard 
the  reception  of  weaker  signals  on  the  main  beam.   The 
conventional  method  of  reducing  this  type  of  interference  by 
adaptive  beamforming  is  often  complex  and  expensive  to  imple- 
ment.  When  the  number  of  spatially  discrete  interference 
sources  is  small,  adaptive  noise  cancelling  can  provide  a 
simpler  and  less  expensive  method  of  coping  with  this  problem. 
Consider  an  array  pattern  with  signal  strengths  and  directions 
as  shown  in  Figure  2.6.   The  array  consists  of  a  circular 
pattern  of  16  equally  spaced  omnidirectional  elements.   The 
outputs  of  the  elements  are  delayed  and  summed  to  form  a  main 
beam  steered  at  a  relative  angle  of  0  degrees.   A  simulated 
"white"  signal  consisting  of  uncorrelated  samples  of  unit 
power  is  assumed  to  be  incident  on  the  beam.   Simulated  inter- 
ference with  the  same  bandwidth  and  with  a  power  of  100  is 
incident  on  the  main  beam  at  a  realtive  angle  of  58  degrees. 
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Fig.  2.6.   Array  Configuration  for  Adaptive 
Sidelobe  Canceller 
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The  entire  array  is  then  connected  to  the  adaptive  noise 
cancelling  system  shown  in  Figure  2.7.   In  this  case  the 
output  of  the  beamformer  serves  as  the  canceller's  primary 
input,  and  the  output  of  one  of  the  elements  (#4)  is  arbi- 
trarily chosen  as  the  reference  input.   The  adaptive  can- 
celler uses  14  weights. 

A  number  of  experiments  performed  in  [Ref .  4]  show  that 
the  signal-to-noise  ratio  at  the  system  output  was  found 
after  convergence  to  be  +20  dB.   The  signal-to-noise  ratio 
at  the  single  array  element  was  -20  dB.   This  result  bears  out 
the  equation  shown  in  the  Wiener  solution  (2.35),  that  the 
signal-to-noise  ratio  at  the  system  output  would  be  the 
reciprocal  of  the  ratio  at  the  reference  input,  which  is 
derived  from  a  single  element. 

F.   CANCELLING  60-HZ  INTERFERENCE  IN  ELECTROCARDIOGRAPHY 

A  practical  example  of  cancelling  60-Hz  interference  is 
found  in  electrocardiography.   A  major  problem  which  exists 
in  the  recording  of  electrocardiograms  (ECG's)  is  the  ap- 
pearance of  unwanted  60-Hz  interference  in  the  output.   Vari- 
ous methods  have  been  utilized  to  help  cancel  the  60-Hz 
interference,  including  more  effective  grounding  techniques 
and  the  use  of  twisted  cabling.   Another  method  capable  of 
reducing  60-Hz  ECG  interference  is  adaptive  noise  cancelling 
via  a  system  such  as  that  shown  in  Figure  2.8. 

The  primary  input  is  taken  from  the  ECG  preamplifier  and 
the  60-Hz  reference  is  taken  from  a  wall  outlet.   The  adaptive 
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filter  is  simple,  containing  only  two  variable  weights,  one 
applied  to  the  reference  input  directly  and  the  other  to  a 
version  of  it  shifted  in  phase  by  90  degrees.   The  two 
weighted  versions  of  the  reference  are  summed  to  form  the 
filter's  output,  which  is  then  subtracted  from  the  primary 
input.   A  valuable  advantage  in  the  use  of  an  adaptive  filter 
rather  than  a  fixed  notch  filter  at  60-Hz,  is  that  the  varia- 
ble weights  allow  the  60-Hz  interference  to  change  in  both 
magnitude  and  phase  and  still  realize  effective  cancellation. 

G.   CANCELLING  PERIODIC  INTERFERENCE  WITH  NO  EXTERNAL 
REFERENCE 

In  many  cases  where  a  broadband  signal  is  corrupted  by 
periodic  interference  there  is  no  external  reference  input 
which  is  free  of  the  signal.   If  a  fixed  delay  is  inserted 
in  a  reference  input  drawn  directly  from  the  primary  input, 
as  shown  in  Figure  2.9,  the  periodic  interference  can,  in 
many  cases,  be  cancelled.   A  key  point  is  that  the  delay  must 
be  chosen  to  be  of  sufficient  length  to  cause  the  broadband 
signal  components  in  the  reference  input  to  become  decorre- 
lated  from  those  in  the  primary  input.   Because  of  their 
periodic  nature,  the  interference  components  will  remain 
correlated  with  each  other. 

By  taking  the  output  from  the  LMS  output  instead  of  the 
difference  signal  the  same  system  can  be  used  to  separate 
broadband  interference  from  a  periodic  signal  as  shown  in 
Figure  2.10.   Such  a  filter  is  often  called  an  adaptive 
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Interference  Without  an  External  Reference 


54 


self-tuning  filter.   Additionally,  applications  of  this  filter 
are  utilized  in  the  adaptive  line  enhancer,  a  system  used  for 
detection  of  a  low  level  signal  imbeded  in  noise.   The  trans- 
fer function  of  this  filter  is  the  digital  Fourier  transform 
of  the  impulse  response.   Its  magnitude  at  the  frequency 
of  the  interference  is  very  nearly  one,  the  value  required 
for  perfect  cancellation. 

H.   THE  ADAPTIVE  NOISE  CANCELLER  AS  A  NOTCH  FILTER 

One  of  the  primary  considerations  for  considering  con- 
straints on  adaptive  filtering  systems  stems  from  the  filter's 
use  as  a  notch  filter.   After  an  analysis  of  the  adaptive 
system  as  a  notch  filter  it  was  felt  that  specific  constraints 
could  be  used  to 

a)  shape  the  frequency  characteristics 

b)  produce  a  faster  convergence. 

Before  considering  the  constraining  equations  and  their 
results  it  is  first  helpful  to  analyze  the  adaptive  noise 
canceller  in  its  notch  filter  mode.   Figure  2.11  depicts  a 
single  frequency  adaptive  noise  canceller  with  two  adaptive 
weights.   Analytical  and  experiemntal  results  show  that  if 
more  than  one  frequency  is  present  in  the  reference  input 
then  a  notch  for  each  will  be  formed.   The  primary  input  is 
assumed  to  be  any  type  of  signal — stochastic,  deterministic, 
periodic,  transient,  or  any  combination  of  these.   The 
reference  input  is  assumed  to  be  a  pure  cosine  wave 
C  cos  (w.t  +  <J>)  .   The  primary  and  reference  inputs  are  sampled 
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Fig.  2.11.   Single  Frequency  Adaptive  Noise  Canceller 
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at  the  frequency  of  to  =  2(Pi)/T  rad/sec.   The  reference 
input  is  sampled  directly  yielding  XI (j),  and  after  under- 
going a  90  degree  phase  shift,  also  produces  X2(j).   Assume 
synchronous  sampling. 

A  transfer  function  for  the  noise  canceller  of  Figure  2.8-1 
can  be  obtained  by  analyzing  signal  propagation  from  the 
primary  input  to  the  system  output. 

The  weights  are  updated  in  accordance  with  the  LMS 
algorithm  yielding 


Wl(j+1)   =   Wl(j)  +  2pe(j)Xl(j) 

W2(j  +  1)   =   W2(j)  +  2ye(j)X2(j) 
The  sampled  reference  inputs  are 


XI  (j)   =   C  cos(ojo(j)T  +  <f>) 


X2(j)   =   C  sinUQ  (j)T  +  4>) 


(2.43) 


(2.44) 


Using  signal  flow  diagram  techniques  and  considering  that  the 
error  signal  at  time  j  =  k  is 

e(j)   =   S(j-k)  (2.45) 

the  filter's  impulse  response  at  k  =  0  can  be  given  as 


y(j)   =   2yC2u(j-l)cos(ajn(j)T)  (2.46) 
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where  u(j)  is  the  discrete  unit  step  function.   The  transfer 
function  of  this  path  is 


~      z  (z  -  cosoj-T) 

G(z)       =      2yCZ      -2 1 

z     -  2zcosconT  +  1 

2 
2yC     (z   COS  u)nT    -    1) 

=      -^ - (2.47) 

z      -    2z  cos  (D-.T   +    1 

This  transfer  function  can  be  expressed  in  terms  of  the 
radian  sampling  frequency  u  =  2(Pi)/T  as 

2  -1 

2  n  C    [z  cos  (2iTajnu)       )    -1] 

G(z)       =    «2 ^1 (2.48) 

z      -2z  cos  (2TTcortcj      )    +    1 
0    s 

When  the  feedback  loop  from  the  adaptive  filter  output  to 
the  difference  signal  is  formed,  the  transfer  function  H(z) 
from  primary  input  to  noise  canceller  output  can  be  written 
as 

2  -1 

z  -2z  cos  (27T0)naj   )  +  1 

H(z)   =  -^ ^ ^-^j j  (2.49) 

z  -2(1  -yC  )z  cos  (2TT0J-.O)   )  +1  -2yC 

This  transfer  function  has  the  property  of  a  notch  filter  at 
the  reference  frequency  a>n.  The  zeros  of  the  transfer  func- 
tion are  located  in  the  z-plane  at 


z      =      exp(±j2TTu^uT1)  (2.50) 

0  s 

and  are  exactly  on  the  unit  circle  at  angles  of  ±2tto)qu) 
radians.   The  poles  are  inside  the  unit  circle  at  a  radial 
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2  1/2 
distance  (l-2pC  )  '   from  the  origin.   For  slow  adaptation 

2 
(small  values  of  yC  )  the  angles  of  the  poles  are  almost 

identical  to  the  zeros.   Since  the  zeros  lie  on  the  unit 
circle,  the  depth  of  the  notch  in  the  transfer  function  is 
infinite  at  frequency  to  =  u  .   The  sharpness  of  the  notch  is 
determined  by  the  closeness  of  the  poles  to  the  zeros.   Corres- 
ponding poles  and  zeros  are  separated  by  a  distance  approxi- 

2 
mately  equal  to  uC  .   The  notch  bandwidth  is  given  by 


BW   =   uC2oj  A  (2.51) 


and  the  Q  of  the  notch  is  determined  by  the  ratio  of  the 
center  frequency  to  the  bandwidth. 

Q   =  —j* (2.52) 

yC  us 

The  single  frequency  noise  canceller  is,  therefore,  equiva- 
lent to  a  stable  notch  filter  when  the  reference  is  a  pure 
cosine  wave.   The  depth  of  the  null  is  generally  superior 
to  that  of  a  fixed  digital  or  analog  filter  because  the 
adaptive  process  maintains  the  null  exactly  at  the  reference 
frequency. 
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III.   THE  CONSTRAINED  ADAPTIVE  FIR  FILTER 

In  this  chapter  the  concept  of  the  constrained  adaptive 
filter  is  introduced.   A  literature  search  has  indicated  that 
very  little  research  has  been  done  in  the  area  of  constrained 
adaptive  filters.   Frost  [Ref.  8]  presents  a  constrained  LMS 
algorithm  which  is  capable  of  adjusting  an  array  of  sensors 
in  real  time  to  respond  to  a  signal  coming  from  a  desired 
direction  while  discriminating  against  noises  coming  from 
other  directions.   A  set  of  linear  constraints  on  the  weights 
maintains  a  chosen  frequency  characteristic  for  the  array  in 
the  direction  of  interest. 

In  this  chapter  three  constraint  conditions  are  presented. 
The  first  two  involve  a  constraint  on  the  angle  between  zero 
locations,  so  that  this  angle  remains  constant  while  the  zero 
location  changes  adaptively.   The  constraint  essentially 
maintains  the  pattern  of  the  zeros  while  their  location  is 
shifted.   The  first  approach  is  a  direct  implementation  in 
which  one  of  the  weights  is  changed  adaptively  using  the  LMS 
algorithm  and  the  others  are  slaved  by  the  fixed  angle  formula 
to  the  adaptively  adjusted  weight.   The  second  implementation 
involves  a  cascaded  version  of  the  foregoing  master-slave 
concept.   The  third  constraint  considered  is  a  linear  con- 
straint on  the  weights.   The  approach  presented  uses  a  La- 
grangian  multiplier  formulation  to  augment  the  cost  function 
in  which  the  basic  LMS  adaptive  algorithm  is  applied. 
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A.   FIXED  ANGLE  CONSTRAINT — DIRECT  IMPLEMENTATION 

Consider  a  transversal  filter  using  the  LMS  algorithm 
with  the  constraint  that  the  angular  separation  between  the 
filter  zeros  in  the  z-plane  is  to  be  a  constant  as  set  by 
design  requirements.   As  an  example,  consider  a  fourth  order 
FIR  filter  with  B  =  e   -  6 ..  ,  where  9.  and  9_  are  the  angles 
of  the  zeros.   The  value  of  B  and  the  magnitude  of  the  zeros 
is  to  be  kept  constant  while  the  angles  9,  and  9~  are  to  be 
adjusted  adaptively.   This  specification  serves  to  maintain 
the  shape  of  the  filter  characteristics  in  the  frequency 
domain.   From  the  direct  implementation  of  the  transversal 
filter  shown  in  Figure  3.1  the  system  transfer  function  is 
given  by 


H(z-1)   =   1  +  W1z"1  +  W2z~2  +  W3z~3  +  W4z~4         (3.1) 
=   (z~4) (z4  +  W  z3  +  W2z2  +  W3z  +  W4)        (3.2) 


which  can  be  factored  into  the  form 

2                                    2         2  2 

.T,    x              (z     -  2r,cos8.  z  +  rn  )  (z    -2r~cos9_z  + r0) 
„(-\  N(z)  111 222 

H(z)  "  DTiT  -  ^ 

(3.3) 

where  r,  and  r_  represent  the  radius  of  the  zero  placement 
in  the  z-plane  and  9,  and  92  are  their  respective  angular 
displacement  from  the  axis.   Carrying  out  the  indicated 
multiplication  of  the  numerator  of  (3.3)  yields 
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Fig.  3.1.   Transversal  Adaptive  Filter  -  Direct 
Implementation 
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4  3     2  2   2 

N(z)   =   z  -(2r,cos6  +2r2cos92)z  +  (rl+4r.r2cos8  cose  +r  )z 

2  2  2  2 

-  (2r.r2cos6  +2r  r  cose  )z  +  r-.r2  (3.4) 


Setting  these  terms  equal  to  the  weight  values  of  (3.2) 
yields 


WQ   =   1  (3.5) 

W,   =   -2(r1cos91  +  r2  cosBJ  (3.6) 

2  2 

W2   =   (r,  +  4r  r2  cos  9,  cos  62  +  r2)  (3.7) 

W3   =   (-2r;.r2)  (r,  cos  e2  +  r2  cos  6,)  (3.8) 

W4   =   r^r2  (3.9) 


The  adaptive  LMS  algorithm  must  now  be  constrained  so  that 
the  angle  B  is  given  by 


B   =  B1   -  92  (3.10) 


For  simplicity  let 


r±      =      r2   =   R  (3.11) 


From  (3.10)  it  follows  that 


1(k+l)   =   B  +  92(k+l)  (3.12) 


Using  (3.11)  and  (3.12)  Equation  (3.6)  can  now  be  written  as 
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W1(k+1)   =   -2R(cos(B  + e2(k+l) )  +  cos(62 (k+1) )     (3.13) 

Using  the  trigonometric  identity 

cos (A+B)   =   cos  A  cos  B  -  sin  A  sin  B 
Equation  (3.13)  can  be  written  as 


W1(k+1)   =   -2R(cos  Bcos(62  (k+1)  )  -sin  B  sin  (9  (k+1)  ) 

+  cos(e2(k+l) )  (3.14) 


Combining  terms,  and  defining  the  constants 

K3   =   -2R(cos(B)  +  1)  (3.15) 

K4   =   -2R(sin(B))  (3.16) 

Equation  (3.13)  can  finally  be  written  as 


W1(k+1)   =   K3  cos  (92(k+l) )  -  K4  sin  (62(k+l) )      (3.17) 


This  transcendental  equation  can  be  solved  iteratively  for 
82(k+l)  using  the  value  for  W, (k+1)  obtained  from  the  LMS 
algorithm 


W,(k+1)   =   Wx(k)  +  2yX(k-l)e(k)  (3.18) 


Now  that  e« (k+1)  is  known,  Equation  (3.7)  can  be  solved  for 
W- (k+1) ,  where 

64 


2      2 
W2(k+1)   =   2R  +  4R  cos  (e  (k+1)  +B)  cos  (9  (k+1))   (3.19) 


Recognizing  that  Equation  (3.8)  is  proportional  to  Equation 
(3.6),  the  fourth  algorithm  iteration  can  be  written  as 


W3(k+1)   =   R2W1(k+l)  (3.20) 


From  (3.9)  W.  is  a  constant  value 


W4(k+1)   =   R4  (3.21) 


The  four  equations  (3.18)  through  (3.21)  comprise  an 
adaptive  iterative  algorithm  that  produces  the  direct  analyti- 
cal realization  of  Figure  3.1.   It  is  seen  that  W,  is  changed 
adaptively  and  W?  and  W_  are  slaved  to  W. ,  and  W.  and  W.  are 
constants.   This  approach  yields  two  pairs  of  zeros  which  are 
located  at  radius  R  from  the  origin  of  the  z  plane  and  precisely 
B  degrees  apart.   Figure  3.2  depicts  the  resulting  z-plane 
diagram. 

B.   FIXED  ANGLE  CONSTRAINT — CASCADED  IMPLEMENTATION 

A  cascaded  implementation  of  the  fourth  order  transversal 
filter  is  shown  in  Figure  3.3.   The  transfer  functions  result- 
ing from  Figure  3 . 3  are  given  by 

2  2 

z   -  2r,  cos  9,  z  +  r.. 

H,(z)   =   ± 5 ± ±-  (3.22) 

1  z 

for  the  first  section,  and  for  the  second  section 
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Fig.  3.2.   Z-Plane  Diagram  of  Zeros  Resulting  from 
Transversal  Filter 

IV  indicates  four  poles  located  at  the 
Origin 
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Fig.  3.3.   Cascaded  Transversal  Filter  with  LMS 

Adaptation  of  W, .   Master-Slave  Concept 
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2  2 

z   -  2r~  cos  8  «z  +  r9 

H2(z)   =   =-2 - =■  (3.23) 


The  weight  values  from  these  two  equations  are 


W,   =   -2  r,  cos  0. 


W   =   r 
w2      rl 


W^   =   -2  r2  cos  62 


W   =   r 
w4      r2 


Once  again  the  algorithm  producing  a  set  of  converging 
weights  begins  with  the  LMS  adaptive  equation 


W^k+1)   =   W,(k)  +  2yX(k-l)e(k)  (3.24) 


The  second  weight  is  a  constant  proportional  to 


W2(k+1)   =   R2  (3.25) 


From  (3.22)  the  value  for  W,  can  be  written  as 


W,(k+1)   =   -  2  rxcos  61(k+l)  (3.26) 


Solving  for  9. (k+1) 

-W  (k+1) 
cose1(k+l)   =   — ^ (3.27) 


where  the  value  of  W, (k+1)  is  known  from  (3.24)  and 
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rx  =  r   =  R 


Therefore, 


-W-^k+l) 
81(k+l)   =  Arccosine( ^ )  (3.28) 


Now  the  third  weight  can  be  written  in  the  form  of 


W3(k+1)       =      -2  Rcose2(k+l)  (3.29) 


where 


92(k+l)      =      B   -    e1(k+l)  (3.30) 


with  B  a  given  constant  set  by  the  user  and  e..(k+l)  known 
from  (3.27).   The  fourth  weight  is  proportional  to 


W4(k+1)   =   R2  (3.31) 


The  four  equations  (3.24),  (3.25)  and  (3.29),  and  (3.31) 
comprise  an  adaptive  iterative  algorithm  that  produces  the 
cascaded  realization  of  Figure  3.3.   As  in  the  direct  reali- 
zation, the  solution  of  Equations  (3.22)  and  (3.23)  using  the 
adaptive  weight  solutions  produces  two  pairs  of  zeros  which 
are  located  at  radius  r,  =  r~  =  R  from  the  origin  of  the 
z-plane  and  precisely  B  degrees  apart. 

C.   LINEARLY  CONSTRAINED  WEIGHTS — LAGRANGE  MULTIPLIER  APPROACH 

Consider  now  the  LMS  adaptive  equations  modified  for  a 
different  type  of  constraint,  namely  a  linear  constraint  on 
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the  sum  of  the  weight  vector.   That  is,  the  constraint  that 


W, (k)  +  W0(k)  +  ...  +  W  (k)   =   K  (3.22) 

L2  n 


where  K  is  some  user  defined  constant.   This  constraint  func- 
tion is  adjoined  to  the  square  of  the  error  equation  by  the 
method  of  Lagrange  multipliers  [Ref.  9].   Recalling  from 
Equation  (1.5)  that 


e  (k)   =   d(k)  -  W(k)TX(k) 


the  adjoined  and  modified  least  mean  squared  error  function 
can  be  written  as  the  cost  function  equation 


J(k)   =   [d(k)-W(k)TX(k) ]2  -  A (k)  [W (k) Tu  -  K]       (3.33) 


where 


u  =   [1   1   ...   1]T 


and  X (k)  is  the  scalar  Lagrange  multiplier. 

In  order  to  minimize  the  mean  squared  error  under  the 
given  constraint  consider  the  gradient  of  (3.3  3)  with  respect 
to  the  weights  and  to  the  Lagrangian  multipliers. 


V[J(k)]    =   -2[d(k)-WT(k)X(k)]X(k)  -  X(k)u         (3.34) 


and 
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V[J(k)] x      =   -[W(k)Tu  -  K]  (3.35) 


The  method  of  steepest  descent  can  be  described  by  the  two 
relationships 


W(k+1)      =      W(k)    +k1v[j(k)]  (3.36) 


and 


X  (k+1)      =      X(k)    +k2V[J(k)]x  (3.37) 


Substituing  Equations  (3.34)  and  (3.35)  into  Equations  (3.36) 
and  (3.37)  yields  the  final  weight  and  Lagrange  parameter 
iteration  algorithms  using  the  LMS  approximation 


W(k+1)   =  W(k)  -  2k1e(k)X(k)  -  x(k)u  (3.38) 


and 


X(k+1)   =   X  (k)  -  k2[W(k)Tu  -  K]  (3.39) 


where 


W(k)   =   the  weight  vector  before  adjustment 
W(k+1)   =   the  weight  vector  after  adjustment 
K  =   the  linear  constraint  value 
kn ,k   =   the  scalar  constants  (k,  k2 <  0) 
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A (k)   =   the  Lagrange  parameter  before  adjustment 
\ (k+1)   =   the  Lagrange  parameter  after  adjustment. 

The  resulting  constrained  adaptive  filter  is  shown  in  Figure 
3.4.   Experimental  results  are  presented  in  the  next  chapter 
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Fig.  3.4.   The  Adaptive  LMS  Filter  with  Linear  Constraint 
and  Lagrange  Parameters 
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IV.   SIMULATIONS  AND  RESULTS 

Using  the  Hewlett-Packard  85  (HP-85)  microcomputer 
several  computer  simulations  are  performed  to  demonstrate 
both  the  unconstrained  and  the  constrained  LMS  adaptive 
systems  discussed  in  Chapters  II  and  III.   The  unconstrained 
adaptive  system  utilized  for  computer  simulation  is  the 
system  depicted  in  Figure  2.1.   The  unconstrained  results 
are  obtained  first  with  nine  adaptive  weights  and  then  with 
only  four  adaptive  weights.   The  noise  input  consists  of 
either  the  sum  of  two  sinewaves  or  zero  mean  white  noise 
added  to  a  sinewave.   In  all  unconstrained  cases  the 
reference  input  (the  desired  waveform)  is  a  sinewave.   The 
constrained  adaptive  systems  are  simulated  for  the  linear 
constraint  using  the  Lagrangian  multiplier  and  for  the 
angular  constraint  (separation  of  zeros  in  the  z-plane 
plot)  in  both  cascaded  and  direct  implementations. 

A.   THE  UNCONSTRAINED  ADAPTIVE  NOISE  CANCELLER 

The  adaptive  noise  canceller  shown  in  Figure  2.1  is 
computer  simulated  using  nine  weights,  each  adapted  by  the 
LMS  algorithm.   The  desired  signal  is  a  10  Hz  sinewave 
sampled  at  f  =  128  Hz.   The  noise  input  is  the  sum  of  two 
equal  sinewaves  with  frequency  f  =  10  Hz  and  frequency 
f  =  35  Hz.   The  number  of  samples  is  N  =  128  and  the 
adaptation  constant  is  y  =  K  =  0.1.   The  system  output 
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converges  after  a  short  learning  period  to  the  desired 
signal.   Figures  4.1  through  4.3  depict  these  results. 

A  sample  of  three  of  the  nine  weights  (w_ ,  w  ,  wfi )  is 
shown  in  Figures  4.4  through  4.6  to  illustrate  the  steady- 
state  solution  of  the  adaptive  weights.   After  just  N  =  24 
all  weights  have  settled  to  within  0.1%  of  their  final 
value.   Figure  4.7  is  the  system  error.   It  should  be  noted 
that  a  faster  sampling  frequency  of  f  =  25  6  reduces  this 
error  even  further. 

The  next  simulation  is  similar  to  the  first  with  the 
exception  that  this  system  uses  only  four  weights  instead 
of  the  nine  weights  used  previously.   Again  the  noise  input 
is  the  sum  of  two  equal  sinewaves  (f  =  4  Hz,  f  =  20  Hz)  and 
the  reference  or  desired  waveform  is  a  sinewave  at  f  =  4  Hz. 
The  adaptation  constant  is  Kl  =  0.1,  N  =  128  and  the 
sampling  frequency  for  the  inputs  is  f  =  128  Hz.   Figures  4.8 
and  4.9  show  the  filter  input  and  the  reference  signal.   It 
is  evident  from  the  system  output  shown  in  Figure  4.10  that 
this  system  does  not  track  as  well  as  the  previous  system 
with  nine  weights.   Plotting  weight  #1  of  the  system  through 
time,  as  shown  in  Figure  4.11,  illustrates  the  oscillatory 
nature  of  the  weight  values. 

The  same  unconstrained  simulation  is  repeated  with  the 
noise  input  taken  as  zero  mean  uncorrelated  noise  (generated 
by  the  HP-85  random  number  generator)  summed  with  a  sinewave 
of  frequency  f  =  10  Hz.   Figure  4.12  shows  the  noise  input. 
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Fig.    4.10.      System  Output  -  Four  Adaptive   Weights 
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The  desired  signal  is  the  sinewave  shown  in  Figure  4.13. 
Nine  adaptive  weights  are  utilized.   The  system  sampling 
frequency  is  f  =  256  Hz  and  the  adaptation  constant  is 
Kl  =  0.1.   The  signal-to-noise  ratio  of  the  input  signal 
is  1.64  dB.   Figure  4.14  shows  the  system  output  while 
Figure  4.15  illustrates  the  system  error.   Figure  4.16  shows 
one  of  the  system  weights  plotted  through  time. 

B.   CONSTRAINED  ADAPTIVE  SYSTEM  SIMULATIONS 

The  simulation  of  the  angularly  constrained  adaptive 
systems  shown  in  Figures  3.1  and  3.3  produces  nearly  identical 
results.   Computationally,  the  iterative  solution  required 
for  the  direct  implementation  is  much  slower  than  that  of 
the  cascaded  system.   Since  both  system  outputs  are  virtually 
identical,  those  of  the  faster  cascaded  system  are  shown. 
All  of  the  linearly  constrained  adaptive  results  utilize 
nine  adaptive  weights  and  all  of  the  angularly  constrained 
systems  utilize  four  weights. 

Using  the  cascaded  adaptive  system  shown  in  Figure  3.3 
simulation  results  are  obtained  for  a  noise  input  shown  in 
Figure  4.17  of  two  sinewaves  of  frequencies  f  =  4  Hz  and 
f  =  20  Hz.   The  sampling  frequency  f  =  128  Hz.   The 
desired  signal  shown  in  Figure  4.18  is  a  sinewave  of 
frequency  f  =  4  Hz.   The  adaptation  constant  is  Kl  =  0.1. 
The  zeros  are  desired  at  a  radius  of  1  and  with  a  separation 
of  12  degrees.   The  system  output  tracks  the  desired  signal 
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Fig.  4.13.   System  Reference  Signal 
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as  shown  in  Figure  4.19  and  places  the  system  zeros  with 
the  exact  requested  degree  of  separation  as  shown  in 
Figure  4.20.   Zero  1  is  at  radius  1  and  an  angle  of  63.39 
degrees.   Zero  2  is  at  a  radius  of  1  and  an  angle  of  51.39 
degrees.   The  other  two  zeros  are  the  conjugate  pairs  of 
the  first  two. 

The  adapted  weight  number  1  is  plotted  through  time  in 
Figure  4.21.   The  same  oscillating  pattern  is  shown  here  as 
is  depicted  in  Figure  4.11.   It  is  felt  that,  as  in  the  case 
of  Figure  4.11,  using  a  greater  number  of  weights  would 
eliminate  the  noisy  weight  pattern  and  accordingly  produce 
a  cleaner  output. 

Suppose  now  that  the  user  wishes  to  drastically  alter 
the  frequency  characteristics.   The  input  to  the  system 
remains  the  same  (as  shown  in  Figure  4.17)  however  now  the 
desired  angle  of  separation  of  the  zeros  is  130  degrees 
instead  of  the  original  12  degrees.   System  output  is  shown 
in  Figure  4.22,  the  reference  signal  is  shown  in  Figure  4.23, 
and  the  z-plane  zero  location  is  shown  in  Figure  4.24. 
Zero  number  1  is  at  a  radius  of  1  and  at  an  angle  of  158.55 
degrees  and  zero  number  2  is  at  a  radius  of  1  and  an  angle 
of  28.55  degrees,  providing  the  prescribed  130  degrees  of 
separation.   The  remaining  two  zeros  form  the  conjugate  pair. 
The  sampling  frequency  is  changed  to  f  =  25  6  Hz  and  the 
adaptation  constant  Kl  is  changed  to  0.05. 
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The  same  simulations  are  performed  for  two  sinusoids  of 
frequencies  f  =  10  Hz  and  f  =  3  5  Hz.   A  sampling  frequency 
of  f  =  256  Hz  is  used  with  an  adaptation  constant  Kl  =  0.1. 
Figures  4.25  through  4.28  depict  the  system  input  (the  sum 
of  the  two  sinusoids) ,  the  filter  output,  the  desired  signal, 
and  the  z-plane  zero  locations  for  a  desired  separation  of 
2  0  degrees.   Zero  1  is  located  at  a  radius  of  1  and  an  angle 
of  50.2  degrees.   Zero  2  is  located  at  a  radius  of  1  and  an 
angle  of  70.2  degrees.   The  remaining  two  zeros  are  the 
conjugate  pairs.   Figures  4.29  and  4.30  show  the  system  out- 
put and  z-plane  zero  location  for  the  same  system  input  and 
same  reference  but  with  a  desired  zero  separation  of  40 
degrees.   Zero  1  is  located  at  a  radius  of  1  and  an  angle 
of  45.4  3  degrees  and  zero  2  is  located  at  a  radius  of  1  and 
an  angle  of  85.43  degrees.   The  remaining  two  zeros  are 
conjugate  pairs. 

C.   LINEARLY  CONSTRAINED  ADAPTIVE  FILTER  SIMULATIONS 
USING  THE  LAGRANGE  MULTIPLIER  TECHNIQUE 

The  linearly  constrained  LMS  adaptive  filter  shown  in 
Figure  3.4  is  simulated  for  various  values  of  K,  the  linear 
constraint  value.   Two  types  of  system  inputs  are  used — 
random  "white"  noise  and  a  sinusoidal  signal  of  frequency 
4  Hz.   In  each  case  the  desired  signal  is  a  damped  exponen- 
tial with  a  final  value  of  0.2. 

Figure  4.31  depicts  the  filter  output  superimposed  upon 
the  desired  output  of  a  damped  exponential  of  final  value 


102 


FILTER  INPUT 


u 

T    t 

£ 

1 


11 


m 


(I 


i  ft  i  HH   i'lH 


H 


<-ir+ 


M-+ 


a 


» 


=128    SAMPLES 


Fig.  4.25.  •  System  Input  -  Four  Adaptive  Weights 


103 


FILTER  OUTPUT  T  = 1 s 2  5  6 
CRSCRDED  SECTIONS 


A 


\ 


IHHH"H-fMH-V+,,-HH,-'*^HH 


^N=/l2  8  ISfifMPLES/ 

v     V 


Fig.  4.26.   System  Output  -  Four  Adaptive  Weights 
Desired  Zero  Separation  =  20  Degrees 


104 


DESIRED  FILTER  OUT 


i\i  (  i  |  h  I  t  i  y  i  i  mi  i t/i  i  i  \ 


-M- 


? 


\/        V 


1    N *=  1  2 S1)    SfiM 

V  / 


PLH 
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Desired  Zero  Separation  =  40  Degrees 
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0.2.   The  linear  constraint  K  is  arbitrarily  chosen  to  be 
zero.   The  adaptation  constant  is  Kl  =  1  E-6.   The  system 
error  is  shown  in  Figure  4.32.   The  sum  of  the  weights  plotted 
through  time  is  shown  in  Figure  4.33.   The  final  value  of 
the  Lagrange  multiplier,  that  is  A CN)  at  time  N  =  64,  is 
found  to  be  -1.022  E-4.   The  final  value  of  the  sum  of 
weights  is  found  to  be  0.059  compared  with  a  requested  value 
of  zero. 

The  same  system  is  again  simulated  for  a  linear  con- 
straint of  K  =  -0.2.   Figure  4.34  shows  that  the  sinusoidal 
input  converges  to  the  desired  damped  exponential  of  final 
value  0.2.   Interestingly,  the  response  converges  and  then 
begins  to  build  an  oscillation.   This  type  of  response  in 
adaptive  systems  has  been  observed  elsewhere  and  may  be  due 
to  an  arithmetic  precision  problem  [Ref.  10].   Figure  4.35 
shows  the  system  error  and  Figure  4.36  shows  the  sum  of 
weights  graph  plotted  through  time.   The  final  value  of  the 
sum  of  weights  is  found  to  be  -0.2427  compared  with  the 
desired  value  of  -0.2. 

The  simulation  is  again  performed,  now  with  a  "white" 
noise  input  and  the  same  damped  exponential  with  final  value 
of  0.2  as  the  desired  signal.   In  the  first  of  the  three 
noise  input  simulations  the  sum  of  weights  constraint  K  is 
set  to  -0.1;  the  second  simulation  has  K  =  0.2;  the  final 
simulation  uses  K  =  0.5.   Figures  4.37  through  4.42  depict 
the  simulation  results.   In  Figure  4.42  the  sum  of  the 
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weights  overshoots  the  constraint  of  K  =  0.5  but  returns 
to  within  10%  of  that  value  at  N  =  128. 

D.   CONCLUSIONS 

This  thesis  has  investigated  adaptive  noise  cancellation 
techniques  in  detail  with  the  objective  of  implementing 
constrained  adaptive  filters.   Two  basic  types  of  constrained 
adaptive  filters  are  presented — the  fixed  angular  separation 
(direct  and  cascaded  implementations)  and  the  adaptive 
Lagrangian  multiplier  linear  constraint  approach.   All  three 
methods  produce  the  desired  output  within  a  very  close 
tolerance  for  the  examples  tested.   As  expected,  the  direct 
implementation  takes  longer  to  converge  than  the  cascade 
approach.   The  results  are  promising,  however,  several  key 
questions  remain  to  be  investigated.   In  particular  the 
convergence  properties  of  these  constrained  adaptive 
algorithms  need  to  be  studied  in  detail.   In  the  case  of 
the  Lagrangian  multiplier  approach,  if  there  is  an  optimal 
steady-state  solution,  the  modified  LMS  algorithm  should 
find  a  minimum  since  it  is  a  simplified  gradient  technique. 
However  the  LMS  minimum  may  not  be  global.   For  the  case  of 
the  angular  constraint  process,  the  stability  of  the 
adaptive  algorithm  and  its  convergence  also  need  to  be 
examined,  although  the  master- slave  would  seem  to  contain 
inherent  stability. 
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Finally,  the  concepts  of  constrained  adaptive  algorithms 
need  to  be  examined  with  more  sophisticated  examples  using 
a  larger  number  of  weights. 
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